Learning a Warped Subspace Model of Faces with Images of Unknown Pose and Illumination

In this paper we tackle the problem of learning the appearances of a person’s face from images with both unknown pose and illumination. The unknown, simultaneous change in pose and illumination makes it difficult to learn 3D face models from data without manual labeling and tracking of features. In comparison, image-based models do not require geometric knowledge of faces but only the statistics of data itself, and therefore are easier to train with images with such variations. We take an image-based approach to the problem and propose a generative model of a warped illumination subspace. Image variations due to illumination change are accounted for by a low-dimensional linear subspace, whereas variations due to pose change are approximated by a geometric warping of images in the subspace. We demonstrate that this model can be efficiently learned via MAP estimation and multiscale registration techniques. With this learned warped subspace we can jointly estimate the pose and the lighting conditions of test images and improve recognition of faces under novel poses and illuminations. We test our algorithm with synthetic faces and real images from the CMU PIE and Yale face databases. The results show improvements in prediction and recognition performance compared to other standard methods.